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WG  Objectives 

•  Identify  examples  of  analysis  tools  and  methods  successfully  used  to 
quantify  the  effectiveness  of  C2  systems. 

•  Identify  analysis  methods  applicable  to  phases  of  the  acquisition  cycle. 

•  Report  findings  and  actionable  recommendations  for  promoting  adequate 
analysis  of  C2  systems  to  inform  acquisition  decisions. 
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WG  Approach 

•  What  are  the  decisions  that  need  to  be  informed  during  the  acquisition 
process? 

•  What  types  of  analyses  need  to  be  performed  to  inform  these  decisions? 

•  What  tools  (models,  simulations,  etc.)  can  be  used  to  perform  these 
analyses? 


Unclassified 


4 


C2 


Command 
&  Control 


SflBlOll 


.  .  cicOC 


Joint  Framework  for 

Measuring  C2  Effectiveness 

23-26  January  2012  |  JHU  Applied  Physics  Lab,  Laurel,  MD 


Metrics 

Derived 
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Analyses  Metrics 


•  Cost 

•  System  Performance 

•  Risk 

•  Cost 

•  Schedule 

•  Technical 

•  Trade  Space 


Timeliness  (time  of  arrival,  latency) 

Accuracy  (data,  representation) 

Completeness 

Range 

Robustness 

Capacity 

Cross  Domain  Information 

Responsiveness 

Architecture 

Scalable 

Trainability/Usability  (ease  of  use) 
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Four  Major  Decision  Points 

•  Materiel  Decision  Document  (MDD) 

•  Milestone  (MS)  A 

•  MSB 

•  MS  C 
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MDD  Tools/Methods/MS&A 

•  SME  Panel 

•  Break  mission  down  to  tasks  and  supporting  capabilities/assets 

•  Map  capabilities  to  task  sets,  available  assets  to  capabilities 

•  Break  capability  gaps  into  required  attributes 

•  DOTmLPF  assessment 

•  Map  available  and  developmental  technologies  to  capability 
gaps/tasks 


MDD  MSA  MSB  MS  C 
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MS  A  T ools/Methods/M&S 

•  Item-Level  M&S 

•  SME  Panels 

•  Qualitative  Assessments 

•  Vendor  Specifications 


MDD  MSA  MSB  MS  C 
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MS  B  T ools/Methods/MS&A 

•  Engineering-  and  Item-level  M&S 

•  Component  Testing 

•  Prototype  System  Demonstrations 

•  Experimentation 


MDD  MSA  MSB  MS  C 
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MS  C  T ools/Methods/MS&A 

•  Forward  Operational  Assessments  (FOA) 

•  Developmental/Operational  Testing  with  Military  Personnel 

•  Physics/Engineering-level  M&S 


MDD  MSA  MSB  MS  C 
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WG  2  Findings 

•  C2  M&S  capability  is  inadequate. 

•  Concerns 

•  Integration  and  performance  of  C2  systems  within  context  of  other 
systems  is  not  addressed. 

•  C2  capability  gaps  are  not  re-assessed  at  each  milestone  decision 
to  ensure  that  product  is  still  relevant. 

•  Are  the  traditional  C2  measures  still  relevant  and  agile  enough  to 
answer  current  and  future  C2  questions? 
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WG  2  Summary 

■  Developed  set  of  metrics  that  can  be  used  to  assess  system 
performance. 

■  Developed  set  of  analytic  methods  and  tools  to  address  system-level 
metrics  at  each  milestone. 


■  Discussed  need  for  system-of-system  and  mission  effectiveness  metrics 
within  acquisition  process. 
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Final  Thought 

•  Quicker  timelines  for  fielding  C2  systems. 


Unclassified 


15 


Joint  Framework  for 

Measuring  C2  Effectiveness 

23-26  January  2012  |  JHU  Applied  Physics  Lab,  Laurel,  MD 


Unclassified 


BACK-UPS 


Unclassified 


16 


M  Command 
W&  &  Control 

,.,,**101 


Joint  Framework  for 

Measuring  C2  Effectiveness 

23-26  January  2012  |  JHU  Applied  Physics  Lab,  Laurel,  MD 


Unclassified 


Analyze 

SeJec? 

Control 

Evaluate 

Concept 

Development 

Materiel 

Solution 

Analysis 

Technology 

Development 

Engineering  &  Manufacturing 
Development  &  Demonstration 

Production 

& 

Deployment 

Operations 
&  Support 

Capability 

Based 

Assessment 


Noieiitl 
Development 
n^rnurm 


MS  A  9  gig 

9  1A  StNle^y 

10  Modular 

11 


MS  B 


1  Core  F  urCItOB 

Ntfd  lo  bo  Performed  by  Gov 

2  OidCQnMhbOMdPerfaiftiirvct 

3  BPR  L  Serve  mg 

A  f»Vj  Fruits  Or  GOv  better 


6  EA 

7  ACqgif  Itsqfi  Msi^ujs'. 
12  £k*tr  Pi  ft  Plan 


MSC/FSDD 

12  Pifi 

■OTSE  IOC  FOC 

i  i  *  I  - 

<§>  t 

FRP  MDA 

Decision 

Review 


15  Oct  03 


CCA  in  integrated  Framework 


Unclassified 


17 


Joint  Framework  for 

Measuring  C2  Effectiveness 

23-26  January  2012  |  JHU  Applied  Physics  Lab,  Laurel,  MD 


Decision 

Info  Needed 

Tools /Methods /M&S 

MDD 

Requirement 

Materiel 

Technology 

Cost  estimate 

l 

MSA 

AoA  has  to  be 
performed  (technology 
based) 

Cost  assessment 

2 

MSB 

Trade  Space  analysis 

3 

MS  C 

AoA  to  inform  Source 
Selection 

4 

Unclassified 


18 


Joint  Framework  for 

Measuring  C2  Effectiveness 

23-26  January  2012  |  JHU  Applied  Physics  Lab,  Laurel,  MD 


•  MDD 

•  Questions  to  be  answered 

•  Does  a  requirement  or  gap  exist? 

•  Does  closing  the  gap  require  a  materiel  solution? 

•  Is  there  a  feasible  technology  to  close  the  gap? 

•  What  would  such  a  technology  cost? 

•  Tools/Methods/MS&A 

•  SME  Panel 

•  Break  mission  down  to  tasks  and  supporting  capabilities/assets 

•  Map  capabilities  to  task  sets 

•  Break  capability  gaps  into  required  attributes 

•  DOTmLPF  assessment 

•  Map  available  and  developmental  technologies  to  capability  gaps/tasks 

•  Metrics 

•  Gap  criticality 

•  Gap  size 

•  Technical  feasibility/schedule  (TRL) 

•  Projected  cost 
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•  MSA 

•  Questions  to  be  answered 

•  What  attributes  are  capability  /  mission  effectiveness  drivers? 

•  What  attributes  are  cost  drivers? 

•  What  potential  technology  solutions  are  worth  pursuing? 

•  How  much  will  such  a  system  cost? 

•  Tools/Methods/M&S 

•  Operational  considerations: 

•  Individual  mission  threads  via  scenario  simulation  (specific  parameters) 

•  Integrated  scenario  via  military  utility  analysis  simulation  (system  performance  in 
context  of  other  systems  and  total  mission) 

•  Cost  analysis: 

•  Analogies  /  SME  inputs 

•  Monte  Carlo  simulation 

•  Presentation 

•  Graphics 

•  Statistical  Analysis 

•  Link  capability  drivers  to  cost  drivers 

•  Metrics 

•  Cost 

•  Base  case  vs  alternatives  performance/effectiveness 

•  Gap  fulfillment  by  alternatives 

•  Uncertainty  analysis 
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•  MS  B 

•  Questions  to  be  answered 

•  Is  this  capability  still  needed  and  will  it  fulfill  gaps? 

•  What  are  the  trade  offs  for  this  capability? 

•  Decision  Metrics 

•  Mission  effectiveness 

•  System  performance 

•  Cost 

•  Uncertainty 

•  Tools/Methods/MS&A 

•  T rade  space  analysis 

•  M&S  can  be  used  with  experimentation  depending  on  the  fidelity  of  the 
technical  maturity  of  the  system  under  analysis  and  available  data 
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•  MS  C 

•  Questions  to  be  answered 

•  Revalidate  the  requirement,  is  it  still  needed? 

•  Do  the  KPPs,  KSAs,  etc.  reflect  operational  relevance? 

•  Is  there  already  fielded  equipment  that  is  a  viable  alternative? 

•  What  value  has  the  materiel  solution  demonstrated  in  DT,  Customer 
Test,  LUT,  etc.? 

•  Decision  Metrics 

•  “Reliability  growth  curve”  for  C2  effectiveness 

•  With  NIE  like  events,  user  data  on  system  performance/effectiveness 

•  Interoperability  with  other  systems 

•  Tools/Methods/MS&A 

•  Network  Integration  Evaluation  (NIE) 

•  Forward  Operational  Assessments  (FOA) 

•  DT  with  military  personnel 
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•  Models 

•  OPNET,  QualNET 

•  T&E 

•  SME  input 

•  Deterministic,  Stochastic,  utilization  models 

•  Parametric  regression 

•  Statistical  analysis 
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